= : - beg CT. 

oe ¥ GOS - 

re eth ee DPSEBOO2Z69RQAFADALADONL-Koency DX 
z 2 5] Office of the Deputy Director for Intelligence 


2 : 7 MAR 884 
si 


MAR 
* 1984 


ae | Sg. 
a Js Let 
Spend. ee MreBe. - 


ee 4 pan 


Approved For Release 2005/12/14 : CIA-RDP86B00269R001100110001-6 


ss Approved For Release SEGREh4 : CIA-RDP86B00269R001100110001-6 


MEMORANDUM FOR THE RECORD 
SUBJECT: INTELLIGENCE SUCCESSES AND FAILURES 


I have been engaged in the following activities, in response to 
the DCI's request that analysts be exposed to the lessons from 
the CIA record regarding intelligence successes and failures. 


I have been teaching a two-hour unit on the subject to a large 
number of analysts -- mostly new hires. I have presented the 
unit to the New Analysts Course and to the Analysis Training 
Course, I have also been presenting it to all DO Career 
Trainees. The unit was also presented to the Military 
Analysis Course and I will soon present it to new branch 
chiefs (Introduction to Supervision). 


This effort, in my view, has worked well in sensitizing the 
students to the challenge of intelligence analysis, the 
pitfalls, and the need to take seriously the recommendations 
of the DCI and the DDI for improved analysis (such as 
challenging assumptions and projecting multiple outcomes). 


I am also prepared to present a short version of the course to 
interested special groups. For example, I will present a 
three-hour_version to the staff of the Operations Center 

5X1 at their[_——sJConference. 


The first running of the 6-day course for analysts is now at 
its midpoint. The syllabus and the annotated reading lists 
are attached. 


The first course has 18 students: 15 from the DI, 1 from 
NIC/AG, 1 from DDST/ORD, and 1 from O/DCI (Historical Staff). 


The student response so far -- as I judge it and as judged 
by OTE colleagues who have sat in on individual sessions -- 
has been quite positive. After two days of discussion on 


the general causes of failure (including a presentation by 
25X14 [leased on the recent SRP study for the DCI), 
the students presented their case studies on failure. 


Task groups of 3 students each presented to the class 
what they think caused the failure in Cuba in 1962, the Middle 
East War of 1973, and Iran, 1978-79. 
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Their presentations were based on readings for the course, 
including selections from post mortems, and on interviews 
with participants in the failure. On Iran, for example, 


the students interviewed e responsible OPA 25X1 
Division Chief at the time, Chief of the Iran 25X11 
Analytical Center, and John Helgerson, then the Assistant 

NIO. 


The student presentations. generally confirmed the previous 
class discussions on the causes of failure. Nonetheless, 
the students engaged in the case studies reported that 
they had gained a sharper appreciation of the problems 
facing the analysts than they could have gotten from 

my lectures and class discussion on problems in general. 


The students now understand that failure does not happen 
only to the other analyst. And I think they now are ready 
to take seriously the second half of the course -- a 
cost-benefit analysis of DCI and DDI recommendations for 
improved analysis. 


To supplement this discussion, selected students will 
present their findings on "successes" in the work of 
their own Offices, or from the general record. Among 
the topics chosen are the Andropov succession and 
Nicaraguan export of revolution. 


Though it is too early to reach a judgment on the course 
overall (do the students change enough to justify six 
days away from their desk?), the remark from several 

of them that their Branch and Division Chiefs should 
take the course is a promising sign. 


25X1 


20 March 1984 


SEGRET, 
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INTELLIGENCE SUCCESSES AND FAILURES 
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Analysis Training Branch 


OTE Course Instructor 
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Analysis Training Branch 
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DAY 


2e. 


INTELLIGENCE SUCCESSES AND FAILURES 


COURSE SCHEDULE 


Introduction 
CIA Record 
Interpretations of Intelligence Failure 


Barriers in the International Environment 


Case Study Assignments 


Political and Organizational Barriers 
Analyst-Generated Barriers I 
Analyst-Generated Barriers II 

Senior Review Panel: Causes of Failure 


4 


Case Studies of Failure I 
Case Studies of Failure II 
Summary on Causes of Failure 


Coping with International Barriers 
Analytical Techniques for Dealing with 
Complexity 

Coping with Deception 

Product Evaluation Staff: 

Requirements for Success 


Coping with Political & Organizational 
Barriers 

The Analyst's Responsibility: Lateral 
Growth 

The Analyst's Responsibility: 
Openmindedness 

The Analyst's Responsibility: 
Relationships 


Case Studies of Success I 
Case Studies of Success II 
Conclusions & Student Evaluations 
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TIME 


0900-1030 
1040-1200 
1300-1415 
1425-1550 
1600-1650 


0900-1030 
1040-1200 
1315-1450 
1500-1630 


0900-1150 
1300-1450 
1500-1630 
0900-1015 


1020-1200 
1315-1450 


1500-1630 


0900-1030 
1040-1200 
1300-1430 
1440-1630 
0900-1150 


1300-1430 
1440-1630 
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INTELLIGENCE SUCCESSES AND FAILURES (ISF) 


INSTRUCTIONAL UNITS 


1. Introduction. DCI, DDI, and ATB purposes in sponsoring 
ISF. Review of course objectives. Student introductions and 
objectives in taking the course. Ground rules. Definition of 
terms frequently used in course. 


2. The CIA Record: Perils of Estimating. Where CIA has 
generally done well, and where poorly. Why we have had most 
failures in anticipating military threats, third world 
revolutions, and sharp policy shifts. The perils of 
estimating: being wise is not always sufficient for being 
right. Comparison of the overall CIA record with that of other 
intelligence services, and the performance of other groups in the 
US (economists, Wall Street and sports "handicappers," academic, 
and journalists). 


3. Interpretations of Intellignece Failure. Critical 
explanations from the reading material. Student views. Concept 


of multiple barriers to correct interpretation of "signals" of an 
on-coming crisis or important event (See chart next page). 


4. Barriers in the International Environment. "Noise" 
(event and data overload). Complexity (the difficulty of 
estimating an adversary's moves before he closes his options). 
Paradox of Warning (Japan would not have attacked on 7 December 
1941 if it had perceived US alert). Cultural Differences 
(thinking like the Politboro). Paradox of Risk (the more 
outrageous a risk, the less likely it will be anticipated, the 
less risky it becomes -- as in the missile crisis of 1962). 
Deception (the key to surprise attack). Student examples from 
their current analytical assignments. 


5. Case Study Assignments. The students will select 
prominent cases of intelligence success or failure (e.g., the 
Iran Revolution) to study (readings and interviews) and explain 
to the class (see Instructional Units 10-11, 21-22). 
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Approved BARRIERS. EE Er exe’ 


Signal Barriers Generated By: 


INTERNATIONAL 
ENVIRONMENT 


+complerity of world affairs 
enational cultural differences 
+multiple actors (surrogates) 


POLICYMAKING 
ENVIRONMENT 


tmisperception of adversaries 
+policy momentum 
+wish{ul thinking ("can-doism") 


ORGANIZATIONAL 
ENVIRONMENT 


+nesource Limitations 


+emphasis on current reporting & 
building-block research 


ANALYTICAL 
ENVIRONMENT 


tannogance/Limidity in estimating 


+nesistance to critical review 
& Serious coordination 


tgubstantive overspecialization 
+overemphasis on in-box 


+denignation of policymakers 
& other non-analysts 


SUCCESS 


NCE 
-RDP86B00269R001100110001-6 


Signal Barriers Generated By: 


INTERNATIONAL 
ENVIRONMENT 


tnoise (data § event overload) 
+deception 6 concealment 
+propaganda 


POLICYMAKING 
EMVIRONMENT 


+bureaucnatic competition: 
threat inflation/deflation; 
non-sharing of information 


ORGANIZATIONAL 
ENVIRONMENT 


+mixed management signals 
(take risks vs. play safe) 
+tunrg battles (nescstance to 
(coordination & off{-Line units) 


ANALYTICAL 
ENVIRONMENT 
+biases (mirror dmaging) 
+prejudice towards: 
continuity of trends; 
confining evidence 


enesistance to structured 
analytical methods 


skills Limitations (writing) 


INTELLIGENCE FATLURE 


¢ 


A a 
Barriers ane defined as obstacles to correct interpretation of signals of 


— re 


developments of importance to US interests (threats and opportunities). 
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6. Political and Organizational Barriers. Explanations 
from the reading Material. Historical examples. Student 
explanations and examples. Political and organizational barriers 
in perspective: To what extent do the problems for intelligence 
analysts (e.g. partisan and bureaucratic competition, unclear 
management signals) flow from the inherent character and strength 
of the US political system and of large organizations? 


Ts Analyst-generated Barriers I. DCI and DDI views. 
Explanations from the reading material. Arrogance and Timidity 
(some analysts are too quick to reach judgments; some, too 
slow). Biases (we cannot operate without analytical frameworks 
to simplify reality, but even the best frameworks do not 
represent reality). Paradox of Expertise (in general, the expert 
on nuances of a foreign country is handicapped in anticipating 
Nuncharacteristic" behavior and therefore "sharp" changes). The 
"Lone Ranger" syndrome (negative analyst attitudes toward 
critical review, meaningful coordination, and alternative views 
of non-analysts). Skill limitations (especially writing, as an 
obstacle to integrating useful criticism and handling less likely 
alternatives). Student responses. 


8. Analyst-generated Barriers II. Above discussion 
continued. 


9. Senior Review Panel Group Views on Causes of 
Intelligence Failure. SRP, guest STAT 


lecturer. 


10. Case Studies on Failure 1. Students will discuss ways 
in which case studies validate or require redefinition of prior 
class discussion on general causes of failure. 


11. Case Studies on Failure II. Procedure same as in number 
10. 


12. Summary on Causes of Failure. Students to relate their 
current views on the causes of failure to their present and 
prospective DI assignments: What are the most important barriers 
they face? Does intelligence failure happen only to the other 
analyst? Who has responsibility to overcome obstacles? General 
discussion on how. 


13. Coping with Barriers in the International System. 
Understanding and adjusting to the limitations to intelligence 
analysis: projection of multiple outcomes; using the system 
(collectors, methodologists, warning specialists). 
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14. Analytical techniques for Dealing with Complexity. 
. Distinguishing types of problems. Survey 
or analytica echniques. 


15. Coping with Deception. - Why and 


how deception succeeds. What the country analyst can contribute 
to the efforts of the warning specialists, to improve the record 
of countering deception. Political and military dimensions. 


16. Product Evaluation Staff views on the Requirements for 
Intelligence Success. Helene Boatner, Chief PES, guest lecturer. 


17. Coping with Political and Organizational Barriers. 
(A) Understanding the policymakers' modus operandi (empathy, role 


playing, contact). (B) The need for and the special challenge of 
"customized" analysis. (C) Brief history of intelligence 
organizations: Does structure matter? (D) Recent CIA changes 
(regional organization, Alert Memoranda, NIO warning Meetings). 
(E) Possible lessons from the corporate world (In Search of 
Excellence). 


18. The Analyst's Responsibility: Lateral Growth. The 
cognitive and analytical processes. "Quotations" (analytical 


building blocks). The importance of writing and briefing skills. 


19. The Analyst's Responsibility: Openmindedness. What 


does it mean, and how can it be implemented? Cost-benefit 
analysis of techniques such as "devil's advocacy," "role playing 
the adversary," and emphasis on disproving rather than confirming 
judgments. Projecting alternative outcomes. Communicating 
levels of confidence. Drawing policy implications. 


20. The Analyst's Responsibility: Strengthening 
Relationships. With substantive peers throughout the Agency and 


Community, with policymakers, with outside experts. Cost-benefit 
analysis of taking seriously divergent points of view, including 
those of "worst casers" and "best casers". The art of making 
reviewers, coordinators, and critics work for you. Why the 
consumer is "king.". 


21. Case Studies of Succes I. Student task groups will work 
to refine class discussion on requirements for success. 
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22. Case Studies of Success II. Procedures the same as in 
no. 21. 


23. Conclusion and Student Evaluations. What have we 
learned? Where and why do students agree (disagree) with 
critics? With instructor? What should be done; what can be 
done: by the analyst, by DI management, through training and 
education. Student evaluation of the course. 
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INTELLIGENCE SUCCESSES & FAILURES: 
READING SELECTIONS: VOLUME I 


A NOTE ON THE READING SELECTIONS 
PENG SELECTIONS. 


The number of readings in this volume and in the companion 
Classified volume will place a large, and probably unrealistic, 
demand on the Students. Nonetheless, each selection either 


Please use the annotated Table of Contents to select the 
articles you believe will be of most Value in Strengthening your 
understanding of the Subject and its component issues and 
Problems. Please Pay particular attention to Tabs F and G in 
preparation for the first day of the course. 

Selection of an article for the course does not necessarily 
represent an endorsement of the accuracy of its facts or the 
soundness of its judgments. At times, one article contradicts 
another on the issue or problem under discussion. 


TABLE OF CONTENTS 


TAB A. COURSE SYLLABUS, OBJECTIVES, AND INSTRUCTIONAL UNITS. 
DEFINITION OF FREQUENTLY-USED TERMS. 
STUDENT EVALUATION FORM. 
TAB B. BRIEFS ON THE PROBLEMS AND PERILS OF ANALYSTS 
STAT 


TAB C. BRIEFS ON THE CIA RECORD STAT 


TAB D. THE RECORD ACCORDING TO THE PIKE COMMITTEE (1976). 


A convenient and informative, if not always balanced, 
review of a series of failures, 


TAB E. W.D. HOWELLS, "INTELLIGENCE CRISES" (1983) 


A defense of intelligence analysts and of some frequently- 
criticized practices by an INR veteran practitioner. 
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INTELLIGENCE SUCCESSES AND FAILURES 
READING SELECTIONS: VOLUME II 


25X1 
TABLE OF CONTENTS 
TAB N. REFLECTIONS ON THE CIA RECORD 
| WILLARD MATTHIAS, "HOW TERED -ESTIMATDS-WENT-WRONG,"— -— ——— 
( STUDIES IN INTELLIGENCE, 12:1 (WINTER 1968). 
STAT [_] ON ESTIMATING THE SOVIET STRATEGIC THREAT 


(1982). 
HOWARD STOERTZ, ON THE SOVIET STRATEGIC THREAT (1982). 


STAT articles are indirect responses to 
STAT e article in Volume I[ | 25x1 


TAB O CASE STUDY ON THE CUBAN MISSILE CRISIS, 1962. 
SNIE 85-3-62: THE MILITARY BUILDUP IN CUBA. 
SHERMAN KENT, "A CRUCIAL ESTIMATE RELIVED" (1964). 


A defense of the analytical process that underlay the 
estimate. 


THE CUBAN MISSILE CRISIS: TWO VIEWS OF THE EVIDENCE 
(1964). 


TAB P. CASE STUDY ON THE 1973 MIDDLE EAST WAR 
RICHARD KERR, INFORMAL NOTE ON THE 1973 MIDDLE EAST WAR. 


< A participant on the problem of conflicting evidence. 
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TAB Q. 


TAB R. 


TAB S. 
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CASE STUDY ON IRANIAN REVOLUTION, 1978-79 
EXCERPTS FROM THE OFFICIAL POSTMORTEM. 


academic specialist on misperception 
(Volume 1, Tab F), was the principal author. Probably 
the most authoritative postmortem, with insights into 
the difficulty of anticipating and monitoring a 
revolution. 


DEALING WITH POLITICAL BARRIERS 


ROBERT GATES, "AN OPPORTUNITY UNFULFILLED: THE USE AND 
PERCEPTIONS OF INTELLIGENCE ANALYSIS IN THE WHITE HOUSE 
(1980). 


On overcoming isolation (ours) and suspicion (theirs). 


| "ON ESTIMATING REACTIONS" (1965). 


An old artform that reduces the barrier between 
intelligence and policymaking after a given policy 
course has been determined. Used frequently during the 
Vietnam War. 

ROBERT GATES ON THE ANALYST'S RESPONSIBILITY 

"THE PREDICTION OF SOVIET INTENTIONS" (1973). 


On lateral growth, openmindedness, and other 
requirements for improved analysis 


EXCERPTS FROM DDI REMARKS 13 JANUARY 1983. 
On multiple projections and other correctives. 


EXCERTS FROM DDI NEWSLETTER, 28 JULY 1983 & 12 JANUARY 
1984. 


On relations with the policymaker, multiple projections, 
and lateral growth. 


Approved For Release 2005/1 21a" Gi Rpps6B00269R001 100110001-6 


Y té& 


25X11 


Approved For Release 2005/12/14 : GtAs3DP86B00269R001100110001 fh 7 os 


Dre 
OTE 83-6825 A183 [83/} 


24 October 1983 


MEMORANDUM FOR: Director of Centra] Intelligence 


VIA: Deputy Director of Central Intelligence 
Executive Director 
Deputy Director for Intelligence 
Deputy Director for Administration 
Director of Training and Education 


FROM: a 
nalysis Training Branch, Office of 


Training and Education 


SUBJECT: Intelligence Successes and Failures (ISF): 
. Status Report on Syllabus & Reading List 


REFERENCE: Your Memorandum to the D/OTE, dated 
7 April 1982, Subject: Training in 
Intelligence Analysis 


1. This memorandum is in response to your request for 
& status report on plans for a course for analysts on the 
lessons of intelligence successes and failures. I welcome 


Deputy Director for Intelligence, the Senior Review Panel, 
and the Product Evaluation Staff, as well as with other 
interested parties in the DI and in the Office of Training 


2. The primary objective of the course, as I read your 
memorandum, is to use the CIA record to illuminate the 
causes of failure and the requirements for success--in a way 
that motivates analysts to improve their on-the-job 
performance. In other words, the course will elicit from 
the record (e.g., from " bad estimates and good estimates") | 
the changes in attitudes and practices that will prepare 
analysts to deal more effectively with tough intelligence 
problems, especially anticipation of military attack, 
revolutionary overthrow of key regimes, and Sharp shifts in 25X11 
policy by present and potential adversaries, 7- (00 
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SUBJECT: Intelligence Successes and Failures (ISF); 
Status Report on Syllabus & Reading List 


3. The Tentative Syllabus for Intelligence Successes 
and Failures (ISF) eontains a list of specific course 
objectives and sketches out 36 hours of seminars, exercises, 
and guest speakers to illustrate the planned thrust of the 
course (Attachment A). . 


4. The selection of reading materials for ISF is still 
in an early stage. The annotated preliminary list of case 
Studies and commentaries by veteran practitioners and 
outside critics provides a general view of the range of 
materials under consideration for the course (Attachment B). 


5. Subject to final approval, I Plan to offer the 
course in January and April 1984--to test its effectiveness 
in achieving your objectives. In response to the DDI's 
recommendation, I am also incorporating the essence of ISF, 
as a two-hour unit, in other OTE courses. This already has 
been done for the current running of the Analysis Training 
Course (DI Career Trainees), the New Analyst Course 
(recently-hired DI analysts), and Introduetion to the DI (DO 
Career Trainees). Additionally, I will, in May 1984, 
prepare a memorandum for you and the DDI on the implications 
for management and training of my research findings and 
instructional experience regarding intelligence successes 
and failures. The final goal is to fine tune the ISF course 
and the two-hour unit for regular presentation by OTE 
Starting in July 1984. 


Attachments: 
As Stated 
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Distribution: 
Orig - Addressee, w/atts 
- DDCI, w/atts 
- ExDir, w/atts 
- Exec. Reg. w/o atts 
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DDA, w/atts 
- D/OTE w/atts 
- OTE Reg. w/o atts 
- ATB/IT/OTE, w/att 
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24 October 1983 


MEMORANDUM FOR: Director of Central Intelligence 


VIA: Deputy Director of Central Intelligence 
Executive Director 
Deputy Director for Intelligence 
Deputy Director for Administration 
Director of Training and Education 


nalysis Training Branch, Office of 
Training and Education 
SUBJECT: Intelligence Successes and Failures (ISF); 
. Status Report on Syllabus & Reading List 
REFERENCE: Your Memorandum to the D/OTE, dated 


7 April 1982, Subject: Training in 
Intelligence Analysis 


1. This memorandum is in response to your request for 
a status report on Plans for a course for analysts on the 
lessons of intelligence successes and failures. I welcome 
the opportunity to solicit your suggestions for shaping the 
syllabus and reading list to Support the objectives you had 


Deputy Director for Intelligence, the Senior Review Panel, 
and the Product Evaluation Staff, as well as with other 
interested parties in the DI and in the Office of Training 
and Education (OTE). 


2. The primary objective of the course, as I read your 
memorandum, is to use the CIA record to illuminate the 


that motivates analysts to improve their on-the-job 

performance. In other words, the course will elicit from 

the record (e.g., from " bad estimates and good estimates") | 

the changes in attitudes and practices that will prepare 

analysts to deal more effectively with tough intelligence 

Problems, especially anticipation of military attack, 25X1 
revolutionary overthrow of key regimes, and Sharp shifts in 


- policy by present and potential adversaries, - ia 
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SUBJECT: Intelligence Successes and Failures (ISF): 
Status Report on Syllabus & Reading List 


3. The Tentative Syllabus for Intelligence Successes 
and Failures (ISF) contains a list of specifie course 
objectives and sketches out 36 hours of seminars, exercises, 
and guest speakers to illustrate the planned thrust of the 
course (Attachment A). 


4. The selection of reading materials for ISF is still 
in an early stage. The annotated preliminary list of case 
studies and commentaries by veteran practitioners and 
outside critics provides a general view of the range of 
materials under consideration for the course (Attachment B). 


5. Subject to final approval, I plan to offer the 
course in January and April 1984--to test its effectiveness 
in achieving your objectives. In response to the DDI's 
recommendation, I am also incorporating the essence of ISF, 
as a two-hour unit, in other OTE courses. This already has 
been done for the current running of the Analysis Training 
Course (DI Career Trainees), the New Analyst Course 
(recently-hired DI analysts), and Introduction to the DI (DO 
Career Trainees). Additionally, I will, in May 1984, 
prepare a memorandum for you and the DDI on the implications 
for management and training of my research findings and 
instructional experience regarding intelligence successes 
and failures. The final goal is to fine tune the ISF course 
and the two-hour unit for regular presentation by OTE 
Starting in July 1984. 


25X1 
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